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(5?) A video teleconferencing system includes a 

plurality of terminals (10-1 10-n, 60), a server 

(12, 72) including a storage (30, 75) for storing 
input information, and a network (14, 73) coupl- 
ing the terminals to the server, where the input 
information is supplied from each terminal to 
other terminals from the server. Each terminal 
includes an input part (20, 63) including a first 
input part (24, 70) for inputting voice infor- 
mation and a second input part (26, 71) for 
inputting image information, where the voice 
information and the image information form the 
input information of the input part and the 
second input part generates a front image and 
an oblique image of a certain user of the termi- 
nal to which the input part belongs, a display 
part (18, 62) for displaying images of users of 
the terminals other than the terminal to which 
the display part belongs, and a processing part 
(16, 61) including a first part (36, 64) for accept- 
ing specifying information which specifies a 
terminal, and a second part (40, 65-67) for 
determining a kind of image to be displayed on 
the display part of each terminal other than the 
terminal to which the processing part belongs. 


II 


. _ _ r 



— s 


Q_ 
LU 


Jouv , 18, rue Saint-Denis, 75001 PARIS 


1 


EP 0 516 371 A2 


2 


BACKGROUND OF THE INVENTION 

The present invention generally relates to video 
teleconferencing systems, and more particularly to a 
video teleconferencing system which provides from a 
server a video teleconferencing service to a plurality 
of terminals which are coupled via a network. 

Recently, video teleconferencing systems are 
used popularly. According to the video teleconferenc- 
ing system, a plurality of terminals are coupled via a 
network and console information such as characters, 
voice information and image information are com- 
bined and exchanged among the terminals to enable 
a video teleconferencing. The video teleconferencing 
system is advantageous in that the system frees the 
users from being restricted by the conference loca- 
tion, and for this reason, there are demands to im- 
prove the video teleconferencing system by providing 
smooth progress and presence. 

FIG.1 shows an essential part of an example of a 
conventional video teleconferencing system. In 
FIG.1, only two terminals 200 are shown, and each 
terminal 200 includes a camera 201 and a display 
202. The camera 201 is provided above the display 
202, and the terminal 200 transmits image information 
related to the user by picking up the user on this cam- 
era 201. 

However, according to this system shown in 
FIG.1, it is impossible to achieve eye-to-eye contact 
between users A and B. In other words, in order for the 
user A to send a front image of himself to the user B, 
the user A must shift his eyes from the display 202 and 
look upward towards the camera 201. On the other 
hand, in order for the user B to see the user A, the user 
B must look straight at the display 202. As a result, the 
user B picked up by the camera 201 in this state ap- 
pears on the display 202 on the side of the user A as 
if the user B is looking down, and there is no eye-to- 
eye contact between the users A and B. 

FIG. 2 shows an essential part of another example 
of the conventional video teleconferencing system. In 
FIG. 2, those parts which are the same as those cor- 
responding parts in FIG.1 are designated by the same 
reference numerals, and a description thereof will be 
omitted. In FIG.2, a semitransparent mirror (half mir- 
ror) 203 is provided in front of the display 202 at each 
terminal 200. The camera 201 is provided above the 
semitransparent mirror 203. 

According to this system shown in FIG.2, it is pos- 
sible to realize eye-to-eye contact between the users 
A and B. However, if the video teleconference in- 
volves three or more people, it becomes uncertain 
who is requesting eye-to-eye contact to who. 

Therefore, the conventional video teleconferenc- 
ing systems have problems in that it is either impos- 
sible to achieve eye-to-eye contact between two 
users or, it is uncertain who is requesting eye-to-eye 
contact to who in the case where the eye-to-eye con- 


tact is possible. As a result, it is difficult to ensure 
smooth progress of the video teleconference and pro- 
vide presence at each of the terminals. 

5 SUMMARY OF THE INVENTION 

Accordingly, it is a general object of the present 
invention to provide a novel and useful video telecon- 
ferencing system in which the problems described 

10 above are eliminated. 

Another and more specific object of the present 
invention is to provide a video teleconferencing sys- 
tem comprising a plurality of terminals, a server in- 
cluding storage means for storing input information, 

15 and a network coupling the terminals to the server, 
where the input information is supplied from each ter- 
minal to other terminals from the server. Each of the 
terminals comprise input means including a first input 
part for inputting voice information and a second input 

20 part for inputting image information, where the voice 
information and the image information form the input 
information of the input means and the second input 
part includes means for generating a front image and 
an oblique image of a certain user of the terminal to 

25 which the input means belongs, display means for dis- 
playing images of users of the terminals other than the 
terminal to which the display means belongs, and 
processing means including first means for accepting 
specifying information which specifies a terminal, and 

30 second means for determining a kind of image to be 
displayed on the display means of each terminal other 
than the terminal to which the processing means be- 
longs. According to the video teleconferencing sys- 
tem of the present invention, it is possible for the user 

35 at each terminal to always be aware of who is request- 
ing eye-to-eye contact with who. In addition, it is pos- 
sible for a first user to promptly achieve the eye-to-eye 
contact with a second user if the second user is also 
requesting eye-to-eye contact with the first user. The 

40 eye-to-eye contact compensates for the lack of com- 
munication conventionally caused by the physical dis- 
tance among the users of the terminals, and thus, it 
becomes possible to realize a video teleconference 
which closely resembles a conference which is held 

45 at a single location. 

Other objects and further features of the present 
invention will be apparent from the following detailed 
description when read in conjunction with the accom- 
panying drawings. 

so 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG.1 is a system block diagram showing an es- 
sential part of an example of a conventional video 
55 teleconferencing system; 

FIG.2 is a system block diagram showing an es- 
s ntial part of another example of the convention- 
al video teleconferencing system; 
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FIGS. 3 through 8 respectively are system block 
diagrams for explaining six important aspects of 
the present invention; 

FIG. 9 is a system block diagram showing an es- 
sential part of an embodiment of a video telecon- 
ference system according to the present inven- 
tion; 

FIG. 10 is a plan view showing an embodiment of 
the arrangement of a display part and an input 
part shown in FIG.9 together with the user; 
FIG.1 1 is a flow chart for explaining the operation 
of a user image display deciding part shown in 
FIG.9; 

FIG.1 2 is a flow chart for explaining the operation 
of an eye-to-eye contact relationship display de- 
ciding part shown in FIG.9; 
FIG.1 3 shows the virtual user position relation- 
ship of six users and eye-to-eye contact requests 
made at a certain time; 

FIGS. 14 through 16 respectively show the posi- 
tion relationship shown in FIG.1 1 viewed from the 
users A, B and C; 

FIG.1 7 shows the front and oblique images of the 
six users A through F picked up by cameras; 
FIGS. 1 8 through 23 show examples of the display 
positions determined by the deciding parts when 
the input information describes the relationship 
shown in FIG.1 3; 

FIG. 24 is a flow chart for explaining an embodi- 
ment of the operation of the reverse image form- 
ing part shown in FIG.9; 

FIG. 25 is a flow chart for explaining an embodi- 
ment of the operation of the displaying part shown 
in FIG.9; and 

FIG. 26 is a flow chart for explaining an embodi- 
ment of the operation of a step 307 shown in 
FIG.11. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

First, a description will be given of six important 
aspects of the present invention, by referring to 
FIGS.3 through 5. 

FIG. 3 is a system block diagram for explaining a 
first aspect of the present invention. In FIG. 3, a plur- 
ality of terminals 10-1, 10-2, .... and 10-n are coupled 
to a server 12 via a network 1 . Each of the terminals 
10-1, 10-2, .... and 10-n include a processing part 16, 
a display part 18 and an input part 20. The input part 
20 includes a console input part 22 for making a con- 
sole input, a voice input part 24 for making a voice in- 
put, and an image input part 26 for making an image 
input. 

The server 12 includes a storage part 30 for stor- 
ing the input information transmitted from the input 

part 20 of each of the terminals 10-1, 10-2 and 10- 

n. The serv r 12 may also b provided with an editing 


part 28 for editing the input information. Of course, the 
editing may be made in each terminal. According to 
this video teleconferencing system, the server 12 pro- 
vides the input information from each of the terminals 

5 10-1, 10-2 and 10-n to other terminals 10-1, 10- 

2 and 10-n. 

According to the first aspect of the present inven- 
tion, the image input part 26 includes means 32 for 
generating a front image of the user using the terminal 

w to which the image input part 26 belongs (hereinafter 
simply referred to as a terminal user), and means 34 
for generating an oblique image of the user. The proc- 
essing part 16 includes means 36 for accepting input 
of information specifying a user with whom the termi- 

15 nal user wishes to make a conversation, and means 
40 for displaying the images of the users on the dis- 
play part 18 so that the front image of the specified 
user and the oblique images of other users are dis- 
played. 

20 FIG.4 is a system block diagram for explaining a 

second aspect of the present invention. In FIG.4, 
those parts which are the same as those correspond- 
ing parts in FIG.3 are designated by the same refer- 
ence numerals, and a description thereof will omitted. 

25 According to this second aspect of the present in- 

vention, the image input part 26 includes a semitrans- 
parent mirror (half mirror) 41 which is inserted be- 
tween the terminal user and the display part 18, a 
camera 42 for generating a front image of the terminal 

30 user via the semitransparent mirror 41 , and a camera 
44 for generating an oblique image of the terminal 
user. 

FIG. 5 is a system block diagram for explaining a 
third aspect of the present invention. In FIG. 5, those 

35 parts which are the same as those corresponding 
parts in FIG.3 are designated by the same reference 
numerals, and a description thereof will omitted. 

According to this third aspect of the present in- 
vention, the processing part 16 includes means 45 for 

40 displaying images of the other users on the display 
part 18 in a horizontal arrangement depending on a 
virtual user position relationship, and means 46 for in- 
serting the image of the specified user at a predeter- 
mined position in the horizontal arrangement. 

45 FIG. 6 is a system block diagram for explaining a 

fourth aspect of the present invention. In FIG. 6, those 
parts which are the same as those corresponding 
parts in FIGS. 4 and 5 are designated by the same ref- 
erence numerals, and a description thereof will omit- 

50 ted. 

According to this fourth aspect of the present in- 
vention, the input part 20 has the same construction 
as the input part 20 shown in FIG.4, while the process- 
ing part 16 has the same construction as the process- 
55 ing part 16 shown in FIG. 5. 

FIG. 7 is a system block diagram for explaining a 
fifth aspect of the present invention. In FIG. 7, those 
parts which are the same as those corresponding 
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parts in FIGS. 3 and 5 are designated by the same ref- 
erence numerals, and a description thereof will omit- 
ted. 

According to this fifth aspect of the present inven- 
tion, the processing part 16 includes means 36 for ac- 
cepting input of information specifying the user with 
whom the terminal user wishes to make a conversa- 
tion, means 40 for displaying the images of the users 
on the display part 18 so that the front image of the 
specified user and the oblique images of other users 
are displayed, means 45 for displaying images of the 
other users on the display part 18 in a horizontal ar- 
rangement depending on a virtual user position rela- 
tionship, and means 46 for inserting the image of the 
specified user at a predetermined position in the hor- 
izontal arrangement. 

FIG. 8 is a system block diagram for explaining a 
sixth aspect of the present invention. In FIG. 8, those 
parts which are the same as those corresponding 
parts in FIGS. 3 through 5 are designated by the same 
reference numerals, and a description thereof will 
omitted. 

According to this sixth aspect of the present in- 
vention, the input part 20 has the same construction 
as the input part 20 shown in FIG. 4, while the process- 
ing part 16 has the same construction as the process- 
ing part 16 shown in FIG. 7. 

According to the present invention, two kinds of 
images, that is, the front image and the oblique image 
of the terminal user can be input from the image input 
part 20. Hence, it is possible to clarify the eye-to-eye 
contact relationship among the users attending the 
video teleconference by selectively displaying one of 
the two kinds of images at the processing part 16 of 
each terminal 10. 

In addition, when the terminal user requests eye- 
to-eye contact with another user, the image of the 
specified user can be arranged in a region of the dis- 
play part 18 where the eye-to-eye contact is possible. 
Hence, it is possible to promptly realize eye- to- eye 
contact between two users upon request. 

Furthermore, when a first user of a terminal re- 
quests eye-to-eye contact with a second user of an- 
other terminal, the front image of the first user is dis- 
played at the terminal of the second user. On the other 
hand, if no eye-to-eye contact is requested, the obli- 
que image of the first user is displayed at the terminal 
of the second user. Therefore, it is possible to clarify 
who is requesting eye-to-eye contact with who. 

Moreover, whether or not to display the front or 
oblique image of each user and the position where 
each image is to be displayed may be determined 
from the eye-to-eye contact requests from the users 
and the virtual user position relationship which is set 
in advance. For this reason, the eye-to-eye contact re- 
quest made from each user can be grasped imme- 
diately from the images displayed on the display part 
18. 


Next, a description will be given of an embodi- 
ment of the video teleconferencing system according 
to the present invention, by referring to FIG. 9. FIG. 9 
shows one of a plurality of terminals 60 which partici- 
5 pate in a video teleconference, a server 72, and a net- 
work 73 which couple the terminals 60 to the server 
72. 

The terminal 60 includes a processing part 61, a 
display part 62, and an input part 63. The processing 

10 part 61 includes a user selecting part 64, a user image 
deciding part 65, an eye-to-eye contact relationship 
display deciding part 66, a displaying part 67, a re- 
versed image forming part 68, and an information 
transmitting/receiving part 69. The processing part 61 

15 may be realized by a microprocessor, for example. On 
the other hand, the input part 63 includes a console 
input part 69 which makes a console input using a 
mouse or the like, a voice input part 70 for making a 
voice input, and an image input part 71 for making an 

20 image input. 

The server 72 includes a storage 75 and an infor- 
mation transmitting/receiving part 79. The storage 
part 75 stores input information transmitted from each 
terminal 60, where the input information includes 

25 character information, voice information, image infor- 
mation and the like. On the other hand, the informa- 
tion transmitting/receiving part 79 is connected to the 
information transmitting/receiving part 69 of the termi- 
nal 60 and exchange information between the termi- 

30 nal 60 and the server 72. The information transmit- 
ting/receiving part 79 of the server 72 is also connect- 
ed to the network 73. 

In the input part 63, the console input part 69 ac- 
cepts the console information input such as charac- 

35 ters, and the voice information input part 70 accepts 
the voice information input. In addition, the image in- 
put part 71 inputs the front image and the oblique im- 
age of the terminal user by some means. 

FIG. 10 is a plan view of an embodiment of the ar- 

40 rangement of the display part 62 and the input part 63 
shown in FIG. 9 together with the user. In FIG. 10, the 
image input part 71 includes a semitransparent mirror 
(half mirror) 76, a camera 77 for picking up a front im- 
age, and a camera 78 for picking up an oblique image. 

45 The semitransparent mirror 76 is provided between 
the display part 12 and the user at an angle of 45° 
when viewed from the user. The camera 77 is ar- 
ranged on the side of the user at an angle of 90°, and 
picks up the front image of the user via the semitrans- 

50 parent mirror 76. On the other hand, the camera 78 is 
arranged on the side of the user at an angle other than 
90°, and picks up the oblique image of the user via the 
semitransparent mirror 76. The information from each 
terminal 60 is supplied to the server 72 and is stored 

55 in the storage part 75. 

As shown in FIG. 9, the user selecting part 64 ac- 
cepts from the terminal user a selection input of the 
user name of the other terminal to whom the eye-to- 
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eye contact is requested. For example, the console in- 
put part 69 includes a keyboard (not shown), and the 
user selecting part 64 receives the selection input 
from the keyboard. The user image display deciding 
part 65 decides the kind of image of each user and the 5 
display position of each user on the display part 62 de- 
pending on the selection (eye-to-eye contact request 
of all users) at each user electing part 64 of the termi- 
nals 60 and depending on the user virtual position re- 
lationship which is initially set in the storage part 75. w 

Furthermore, the eye-to-eye contact relationship 
display deciding part 66 fins a pair of users in an eye- 
to-eye contact relationship, and decides the link of the 
images and the use of the right-to-left reversed image. 
In other words, the pair of users presently in the eye- 15 
to-eye contact relationship is found from the eye-to- 
eye contact requests made of all of the users, and by 
displaying the right-to-left reversed image and the 
link, the display of the eye-to-eye contact relationship 
is decided. On the other hand, the displaying part 67 20 
receives the decision from the deciding part 66 and 
displays the images of the users on the display part 
62 by transferring thereto the front image, the oblique 
image and/or the oblique reversed image of each user 
from the storage part 75 and/or the reversed image 25 
forming part 68. The required image information is 
transferred from the storage part 75 and the reversed 
image forming part 68. 

The reversed image forming part 68 receives in- 
structions from the decid ing parts 65 and 66, transfers 30 
the oblique image from the storage part 75, and forms 
the right-to-left reversed image related to the oblique 
image of the user. 

Therefore, the terminal user can request eye-to- 
eye contact with a desired user, and the eye-to-eye 35 
contact is achieved if the desired user also request 
eye-to-eye contact with the terminal user. In addition, 
the eye- to- eye contact relationships of other users 
can also be observed from the display. Further, be- 
cause the virtual user position relationship does not 40 
change among the users, it becomes possible to 
make conversion during the video teleconference as 
if all of the participating users are present at the same 
conference room. 

Next, a description will be given of the operation 45 
of this embodiment. 

FIG.1 1 is a flow chart for explaining the operation 
of the deciding part 65. The deciding part 65 decides 
the user which is to be displayed in the region (here- 
inafter referred to as the E-T-E region) where the eye- so 
to-eye contact is possible and the kind of image to be 
displayed on the display part 62. 

First, a step 300 decides whether or not the ter- 
minal user is requesting the eye-to-eye contact with 
a certain user of another terminal 60. If the decision 55 
result in the step 300 is YES, a step 301 decides 
whether or not the certain user is also requesting the 
eye-to-eye contact with the terminal user. If the deci- 


sion result in the step 301 is YES, a step 302 displays 
the front image of the certain user in the E-T-E region 
of the display part 62. On th other hand, if the deci- 
sion result in the step 301 is NO, a step 303 displays 
the oblique image of the certain user in the E-T-E re- 
gion of the display part 62. 

If the decision result in the step 300 is NO, a step 
304 decides whether or not a user of other terminal 60 
is requesting the eye-to-eye contact with the terminal 
user. If the decision result in the step 304 is YES, a 
step 305 displays the front image of this user in the E- 
T-E region of the display part 62. A plurality of users 
may be requesting the eye-to-eye contact with the ter- 
minal user, but in such a case, the priority with which 
the images of the users are displayed is determined 
time-sequentially depending on the time of request. 
On the other hand, if the decision result in the step 304 
is NO, a step 306 displays in the E-T-E region the ob- 
lique image of a user which is determined in advance 
by the initial setting. 

Next, the images to be displayed in the regions on 
the periphery of the E-T-E region of the display part 
62 are determined. A step 307 determines the display 
region (position) of the images related to each of the 
users other than the certain user which is to be dis- 
played in the E-T-E region based on the virtual user 
position relationship which is determined in advance 
by the initial setting. 

Then, a step 308 decides whether or not each of 
the users of the other terminals 60 excluding the cer- 
tain user which is to be displayed in the E-T-E region 
are requesting the eye-to-eye contact with the termi- 
nal user. A step 309 displays the front image in one 
of the peripheral regions if the decision result in the 
step 308 is YES. On the other hand, if the decision re- 
sult in the step 308 is NO, a step 310 decides whether 
or not each user is to be displayed on the reference 
side (for example, left side) with respect to the E-T-E 
region. If the decision result in the step 310 is YES, a 
step 311 displays the oblique image of the user. On 
the other hand, a step 312 displays a right-to-left re- 
versed oblique image of the user if the decision result 
in the step 310 is NO. Hence, in either case, the obli- 
que image of the user is displayed if the decision re- 
sult in the step 308 is NO. 

FIG. 12 is a flow chart for explaining the operation 
of the deciding part 66. 

In FIG. 12, a step 400 decides whether or not 
there is a pair of users for which the eye-to-eye con- 
tact relationship presently exists out of all of the users 
of the terminals 60. The process ends if the decision 
result in the step 400 is NO. But if the decision result 
in the step 400 is YES, a step 401 reverses the right 
and left sides of the oblique image related to one of 
the users forming the pair and determines the display 
so that the pair of users face each other. In addition, 
a step 402 displays the link of the images related to 
this pair so that the relationship is easier to see. 
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Next, a description will be given of the video tel- 
econference which is made in this embodiment 
among six users for a specific virtual user position re- 
lationship. 

FIG. 13 shows the virtual user position relation- 
ship of the six users and the eye-to-eye contact re- 
quests made at a certain time. In the state shown in 
FIG. 13, the eye-to-eye contact relationship stands 
between the users A and D. In addition, no user is 
making the eye-to-eye contact request with respect to 
the user E. 

FIGS. 14 through 16 respectively show the posi- 
tion relationship shown in FIG. 11 viewed from the 
users A, B and C. The image of the user displayed in 
the E-T-E region changes depending which user 
makes the eye-to-eye contact request to which user, 
but the images of the users displayed in the peripheral 
regions to the E-T-E region are determined as if a 
turntable rotates as may be seen from FIGS. 14 
through 16. In other words, the relative position rela- 
tionship of the users is always the same. 

FIG.1 7 shows the front and oblique images of the 
six users A through F picked up by the cameras 77 
and 78. FIGS. 18 through 23 show examples of the 
display positions determined by the deciding parts 65 
and 66 when the input information describes the rela- 
tionship shown in FIG. 13. In FIGS. 18 through 23, 7F" 
denotes the front image, 70" denotes the oblique im- 
age, and "/OR" denotes the oblique reversed image. 

FIG.1 8 shows the display part 62 of the user A, 
and the six users A through F are participating in a vid- 
eo teleconference related to the marketing plan for a 
new product. In this case, the E-T-E region for dis- 
playing the image of the user with whom the eye-to- 
ye contact relationship stands is located at the upper 
central part of the display part 62. In this example, the 
front image of the user D which is in the eye-to-eye 
contact relationship with the user A is displayed in this 
E-T-E region. The oblique images of the users E and 
F are displayed on the right of the E-T-E region based 
on the virtual user position relationship of FIG.1 3. On 
the other hand, the oblique reversed images of the 
users B and C are displayed on the left of the E-T-E 
region based on the virtual user position relationship 
of FIG.1 3. 

FIGS. 19 through 23 respectively show the upper 
parts of the display parts 62 of the users B through F. 

The user B is requesting the eye-to-eye contact 
with the user E. Hence, the image of the user E is dis- 
played at the central part, that is, in the E-T-E region 
of the display part 62 of the user B, in FIG.1 9. How- 
ever, since the user E is not requesting the eye-to-eye 
contact with the user B, the oblique image of the user 
E is displayed in the E-T-E region. On the other hand, 
although the image of the user C is displayed on the 
left of the E-T-E region based on the virtual user pos- 
ition relationship of FIG. 13, this user C is requesting 
the eye-to-eye contact with the user B, and for this 


reason, the front image of the user C is displayed. In 
addition, because the users A and D are in the eye- 
to-eye contact relationship, the lower parts of the cor- 
responding images are linked by a line to indicate this 

5 relationship. 

The user C is requesting the eye-to-eye contact 
with the user B. Hence, the image of the user B is dis- 
played in the E-T-E region of the display part 62 of the 
user C in FIG.20. 

w In FIG. 21 , the front images of the users A and F 

are displayed on the display part 62 of the user D. Be- 
cause the time when the user A made the eye-to-ey 
contact request with respect to the user D is earlier 
than the time when the user F made the eye-to-eye 

15 contact request with respect to the user D, the front 
image of the user A is displayed in the E-T-E region. 

The user E is not making the eye-to-eye contact 
request to any of the users A through D and F. For this 
reason, the reference user which is determined in ad- 

20 vance by the initial setting should originally be dis- 
played in the E-T-E region of the display part 62 of the 
userE. From the relationship shown in FIG. 16, the im- 
age of the user C arranged on the opposite end from 
the user E is regarded as the reference user, for ex- 

25 ample. But in this particular case, the user B is making 
the eye-to-eye contact request to the user E, and for 
this reason, the front image of the user B is displayed 
in the E-T-E region of the display part 62 as shown in 
FIG. 22. In this case, if the user E notices that the user 

30 B is requesting the eye-to-eye contact with respect to 
the user E and the user E requests the eye-to-eye 
contact with respect to the user B in response thereto, 
the eye-to-eye contact relationship stands. 

The user F is making the eye-to-eye contact re- 

35 quest with respect to the user D. For this reason, the 
image of the user D is displayed in the E-T-E region 
of the display part 62 of the user F as shown in FIG.23. 
Because the users D and A are in the eye-to-eye con- 
tact relationship, the oblique image of the user O is 

40 displayed in the E-T-E region. Furthermore, in order 
to facilitate the user F from understanding the eye-to- 
eye contact relationship between the users D and A, 
the oblique images of the users D and A are displayed 
so as to face each other. Thus, the oblique reversed 

45 image of the user D is displayed in the E-T-E region. 

Next, a description will be given of an embodi- 
ment of the operation of the reverse image forming 
part 68 shown in FIG.9, by referring to FIG.24. 

In FIG.24, a step S1 reads a target image which 

so is the su bject of the d isplay. A step S2 prepares there- 
in a memory region having a memory capacity identi- 
cal to the memory capacity required to store the target 
image. A step S3 measures the size S of the target im- 
age in terms of the number of pixels arranged from the 

55 left to right of the target image. A step S4 initializes a 
variable i to zero. A step S5 inputs the pixel compo- 
nent related to the ith pixel from the left of the target 
image as th pixel component related to the jth pixel 
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from the left of a forming image, where j = S - i. Then, 
a step S6 increments i by one, and a step S7 decides 
whether or not i is less than S. The process returns to 
the step S5 if the decision result in the step S7 is YES, 
but the process ends if the decision resutt in the step 
S7 is NO. The forming image which is formed by this 
process is the reversed image in which the right and 
left sides of the target image are reversed. 

Next, a description will be given of an embodi- 
ment of the operation of the displaying part 67 shown 
in FIG. 9, by referring to FIG.25. 

As described above, the deciding part 65 decides 
which one of the front image, the oblique image and 
the oblique reversed image is to be displayed. Based 
on this decision made in the deciding part 65, the de- 
ciding part 66 decides where the image of each user 
is to be displayed on the display part 62. The display- 
ing part 67 displays the image decided in the deciding 
part 65 at the position decided by the deciding part 66. 

In FIG.25, a step S11 sets a variable N to N = M 
- 1 , where M denotes the number of users. A step S 1 2 
initializes a variable k to zero, and a step S13 incre- 
ments the k by one. A step S14 prepares the display- 
ing image related to the user k. This displaying image 
is the front image, the oblique image or the oblique re- 
versed image decided by the deciding part 65. Then, 
a step S15 displays the displaying image at the posi- 
tion decided by the deciding part 66. A step S16 de- 
cides whether or not k is less than or equal to N. The 
process returns to the step S13 if the decision result 
in the step S1 6 is YES, but the process ends if the de- 
cision result in the step S16 is YES. 

Next, a description will be given of an embodi- 
ment of the operation of the step 307 shown in FIG. 1 1 , 
by referring to FIG. 26. FIG. 26 shows the embodiment 
of the operation of the step 307. 

When carrying out the step 307, the image of the 
user to be displayed in the E-T-E region of the display 
part 62 is already determined by the steps 300 
through 306 shown in FIG.1 1 . Hence, the step 307 de- 
termines the images of the users to be displayed on 
the right and left of the E-T-E region shown in FIG. 1 8, 
for example. 

In FIG.26, a step S21 obtains N1 = (M - 2)/2, 
where the figures of N1 below the decimal point are 
omitted. In this case, the terminal user and the user 
to be displayed in the E-T-E region must be excluded 
from the users to be displayed in the peripheral re- 
gions of the E-T-E region, and this is why 2 is subtract- 
ed from M. A step S22 decides whether or not N1 is 
greater than or equal to 1 . If the decision result in the 
step S22 is YES, a step S23 initializes a variable I to 
zero, and a step S24 increments I by one. A step S25 
determines the display position of the user who is lo- 
cated at the Ith position to the right of the user who is 
to be displayed in the E-T-E region in the virtual user 
position relationship shown in FIG.1 3, to the Ith posi- 
tion to the right of the E-T-E region of the display part 


62. Then, a step S26 decides whether or not I is less 
than or equal to N1. The process returns to the step 
S24 if the decision result in the step S26 is YES. 

On the other hand, if the decision result in the step 
5 S22 or S26 is NO, a step S27 obtains N2 = (M - 2)/2, 
where the figures of N2 below the decimal point are 
omitted. A step S28 decides whether or not N2 is 
greater than or equal to 1 . If the decision result in the 
step S28 is YES, a step S29 initializes a variable I to 
10 zero, and a step S30 increments I by one. A step S31 
determines the display position of the user who is lo- 
cated at the Ith position to the left of the user who is 
to be displayed in the E-T-E region in the virtual user 
position relationship shown in FIG. 13, to the Ith posi- 
15 tion to the left of the E-T-E region of the display part 
62. Then, a step S32 decides whether or not I is less 
than or equal to N2. The process returns to the step 
S30 if the decision result in the step S32 is YES. On 
the other hand, the process ends if the decision result 
20 in the step S28 or S32 is NO. 

In the described embodiments, the display part 
62 includes one screen for displaying the users. How- 
ever, it is also possible to use a plurality of display 
screens to display one user on each display screen, 
25 for example. 

In addition, the display positions of the images of 
the users, including the position of the E-T-E region, 
are not limited to those of the described embodiment. 

Further, the present invention is not limited to 
30 these embodiments, but various variations and mod- 
ifications may be made without departing from the 
scope of the present invention. 

35 Claims 

1. A video teleconferencing system comprising a 

plurality of terminals (1 0-1 1 0-n, 60); a server 

(12, 72) including storage means (30, 75) forstor- 

40 ing input information; and a network (14, 73) cou- 

pling said terminals to said server, said input in- 
formation being supplied from each terminal to 
other terminals from said server, characterized in 
that each of said terminals comprise: 

45 input means (20, 63) including a first input 

part (24. 70) for inputting voice information, and 
a second input part (26, 71 ) for inputting image in- 
formation, said voice information and said image 
information forming the input information of said 

50 input means, said second input part including 

means for generating a front image and an obli- 
que image of a certain user of the terminal to 
which said input means belongs; 

display m ans (18, 62) for displaying im- 

55 ages of users of the terminals other than the ter- 

minal to which said display means belongs; and 
processing means (16, 61) including first 
means (36, 64) for accepting specifying informa- 
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tion which specifies a terminal, and second 
means (40, 65-67) for determining a kind of image 
to be displayed on the display means of each ter- 
minal other than the terminal to which said proc- 
essing means belongs. 5 

The video teleconferencing system as claimed in 
claim 1, characterized in that said second means 
(40, 65-67) of said processing means (16, 61) dis- 
plays on said display means (18, 62) the front im- w 
age of a user of a specific terminal which makes 
an eye-to-eye contact request to the terminal to 
which said second means belongs by the speci- 
fying information. 

15 

The video teleconferencing system as claimed in 
claim 2, characterized in that said second means 
(40, 65-67) of said processing means (16,61) dis- 
plays the front image of the user of the specific 
terminal at a predetermined position on said dis- 20 
play means (18. 62). 


8. The video teleconferencing system as claimed in 
any of claims 1 to 7, characterized in that each ter- 
minal further comprises means (22, 69), coupled 
to said processing means (16, 61 ), for generating 
the specifying information which requests an eye- 
to-eye contact with a user of an arbitrary terminal. 

9. The video teleconferencing system as claimed in 
any of claims 1 to 7, characterized in that said in- 
put means (20, 63) further includes a third part 
(22, 69) for inputting console information, and 
said console information forms the input informa- 
tion together with the voice information and the 
image information. 

10. The video teleconferencing system as claimed in 
claim 9, characterized in that said third part (22, 
69) of said input means (20, 63) is coupled to said 
processing means (16, 61) and also generates 
the specifying information which requests an eye- 
to-eye contact with a user of an arbitrary terminal. 


4. The video teleconferencing system as claimed in 
claim 3, characterized in that said second means 

(40, 65-67) of said processing means (16,61) dis- 25 
plays on said display means (18, 62) the oblique 
image of each user of a terminal other than the 
specific terminal. 

5. The video teleconferencing system as claimed in 30 
claim 4, characterized in that said processing 
means (16, 61) further includes third means (45, 

46, 65-67) for determining a display position of 
each oblique image on said display means (18, 
62) relative to the predetermined position de- 35 
pending on a virtual user position relationship 
which is preset and indicates a virtual position re- 
lationship of the users of said terminals. 


11. The video teleconferencing system as claimed in 
any of claims 1 to 10, characterized in that the 
second part (26, 71 ) of said input means (20, 63) 
includes a semitransparent mirror (76) interposed 
between said display means (1 8, 62) and the cer- 
tain user of the terminal to which said display 
means belongs, a first camera (77) for picking up 
a front image of the certain user via said semi- 
transparent mirror, and a second camera (78) for 
picking up an oblique image of the certain user via 
said semitransparent mirror. 


6. The video teleconferencing system as claimed in 40 
claim 5, characterized in that said processing 
means (1 6, 61 ) further includes fourth means (68) 

for forming an oblique reversed image by revers- 
ing right and left sides of the oblique image, and 
said third means (45, 46, 65-67) determines a dis- 45 
play position of each oblique reversed image on 
said display means (18, 62) relative to the prede- 
termined position depending on the virtual user 
position relationship. 

50 

7. The video teleconferencing system as claimed in 
any of claims 1 to 6, characterized in that said 
processing means (16, 61) further includes 
means (67) for displaying on said display means 

(18, 62) a link between images of two users of ter- 55 
minals requesting eye-to-eye contact with each 
other. 


EP 0 516 371 A2 



9 


4 


EP 0 516 371 A2 



10 


EP 0 516 371 A2 


10- 1 


22 


CONSOLE 
INPUT 


,-24 


VOICE 
INPUT 


IMAGE INPUT 


32 


34 


26 


20 ^18 


DISPLAY 


PROCESSING 


36 40 


16 


J 


L 


FIG. 3 

10-2 


s 22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


IMAGE INPUT 


3 2*34 


26 


20 is 


DISPLAY 


PROCESSING 


36 40 

/ 


16 


J 




SERVER . 




, 30 
~^-28 






r 


IO-n 


22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


IMAGE INPUT 


32 34 


26 


2Q r ia 


DISPLAY 


PROCESSING 


36 IP 
J. 


16 


J 


■ 12 


t 


EP 0 516 371 A2 


r 


IO-I 


22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


•aer IMAGE 
^° ^ INPUT 



1 8 


T 20 


DISPLAY 


PROCESSING 


36 


t 40 


16 


FIG A 

^ 10-2 


r22 



CONSOLE 
INPUT 

, 24 



VOICE 
INPUT 

^ IMAGE 
26 -\ INPUT 


41 44 



m 





K 2Q 


18 


DISPLAY 


PROCESSING 


36 


40 


16 


7 



.30 


10 -n 


s 22 


CONSOLE 
INPUT 

r24 


VOICE 
INPUT 

0 - IMAGE 
^ INPUT 











20 


r 


18 


DISPLA Y 


PROCESSING 


36 4Q 
L 


16 


J 


12 


-28 


12 


EP 0 516 371 A2 


10- 1 


22 


CONSOLE 
INPUT 


r24 


VOICE 
INPUT 


IMAGE INPUT 


32 


34 


26 


20 r i8 


DISPLAY 


PROCESSING 


45 46 

/ 


16 


J 


FIG. 5 


, 10-2 


s 22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


IMAGE INPUT 


32^4 


26 


2Q r l8 


DISPLAY 


PROCESSING 


45 46 

L 


16 


T 



14 


L 


lO-n 


jL 


22 


CONSOLE 
INPUT 


JL 


24 


VOICE 
INPUT 


IMAGE INPUT 


32 34 


26 


20 ,18 


DISPLAY 


PROCESSING 


45 


16 


T 


SERVER 
— , 30 

— ~\~28 


12 


13 


EP 0 516 371 A2 


10- 1 


r22 


CONSOLE 
INPUT 


~2~4 


VOICE 
INPUT 


i*r IMAGE 
^° ^ INPUT 



18 


*20 


DISPLAY 


PROCESSING 


45 46 


16 


7 


FIG. 6 

^ 10-2 


r22 



CONSOLE 
INPUT 

s 24 



VOICE 
INPUT 

IMAGE 
26 ^ INPUT 


41 44 








K 20 


18 


DISPLAY 


PROCESSING 


45 46 


16 


7 


JO-n 


L 


r 22 


CONSOLE 
INPUT 

s24 


VOICE 
INPUT 

~~ IMAGE 
^6 ^ INPUT 



41 44 



m 





20 J 


r 


18 


DISPLAY 


PROCESSING 


45 46 

L 


16 


7 



EP 0 516 371 A2 


, 10- 1 


22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


IMAGE INPUT 


32 


34 


26 


20,18 


DISPLAY 


PROCESSING 


-36 
-40 

-45 
-46 


16 


7 


L 


FIG. 7 

IO-2 


s 22 


CONSOLE 
INPUT 


24 


VOICE 
INPUT 


IMAGE INPUT 


32^4 


26 


20 & 


DISPLAY 


PROCESSING 


[-36 
40 
^45 
J*6 


16 



IO-n 


22 


CONSOLE 
INPUT 


r24 


VOICE 
INPUT 


IMAGE INPUT 


32 3 4 


26 


20 r , a 


DISPLA Y 


PROCESSING 


J36 


AO 


A5 
46 


16 


~I2 


EP 0 516 371 A2 


L 


10- 1 


22 


CONSOLE 
INPUT 


~2~4 


VOICE 
INPUT 


c*r IMAGE 
* ° ^ INPUT 


4U 44 



18- 


T 20 


DISPLAY 


PROCESSING 


Y-36 
40 

-45 
-46 


16 


FIG. 8 

^ 10-2 


r22 



CONSOLE 
INPUT 

,24 



VOICE 
INPUT 

IMAGE 
26 -\ INPUT 


41 44 



m 





k 20 


18 


DISPLAY 


PROCESSING 


36 
40 

,45 

L46 


16 



z 


lO-n 


r 22 


CONSOLE 
INPUT 

s24 



VOICE 
INPUT 

0 _ IMAGE 
26 ^ INPUT 



41 44 

m 








20' 


JL 


18 


DISPLAY 


PROCESSING 


1.36 
40 


^.45 
\-46 


16- 


12 


16 


EP 0 516 371 A2 


5 

° Uj 


/ > 




INFO 



o 



17 


■f, 


EP 0 516 371 A2 



FIG. IO 



TO SERVER 72 


18 


EP 0 516 371 A2 


F/G. // 



304 


^ uTHER 
E-T-E CONTACT 
REQUEST 


YES 


NO 


DlSPLA Y FRONT 
IMAGE IN 
E-T-E REG I ON 


L 


303 


display 
oblique 

IMAGE IN 

E-T-E 

REGION 


L 


305 


DISPLAY 
FRONT 
IMAGE IN 
E-T-E 
REGION 


( 


306 


DISPLAY 
OBLIQUE 
IMAGE IN 
E-T-E 
REGION 


-307 


DETERMINE DISPLAY 
POSITION 



NO 


DETERMINE DISPLAY 
OF FRONT IMAGE OF 
USER MAKING REQUEST 


DISPLAY OBLIQUE 
IMAGE 


312 


DISPLAY 
REVERSED 

OBLIQUE 
IMAGE 


19 


EPO 516 371 A2 


# 


FIG. 12 


C START ) 



REVERSE ONE 
OBLIQUE IMAGE 


I 


JL 


402 


INSTRUCT LINK 
OF PAIR 


Q END ^ 


20 


EP 0 516 371 A2 


FIG. 13 


B 


FIG. 14 


EP 0 516 371 A2 



FIG. 1 5 



FIG. 1 6 

I 



22 



EP 0 516 371 A2 


FIG . 17 


A 


B 


D 


E 


F 












• Vt» Y 1 I 






23 


EP 0 516 371 A2 


FIG. 18 


B/OR 


C/OR 


D/F 


E/O 


F/O 







WITH REGARD TO MARKETING PLAN FOR 
NEW PRODUCT 

GOALS TO BE ACHIEVED ? 
ACHIEVEMENTS FOR FIST HALF OF YEAR 
T AGE TED AGE GROUP " 


24 



EP 0 516 371 A2 


FIG. 19 


C/F D/OR E/O F/O A/O 



FIG. 20 


F/OR A/OR B/O D/O E/O 



25 


0 

EPO 516 371 A2 


FIG. 21 


E/OR 

F/F 

A/F 

B/ 0 


C/ O 


V 1 
y— 







FIG. 2i 

> 




F/OR 

A /OR 

B/F 

C/O 


D/O 



ol 









26 



EP 0 516 371 A2 


FIG. 2 3 

B/OR C/OR D/OR E/O A/O 


27 


4* 


EP 0 516 371 A2 


FIG. 24 


( START ) 




^ SI 

< 

tf£>3£> TARGET 


IMAGE 




PREPARE MEMORY 
REGION 


MEASURE S 


i = O 


r 


52 


S3 


S4 


S5 


INPUT COMPONENT OF i TH PIXEL OF 
TARGET IMAGE AS COMPONENT OF j TH 
PIXEL OF FORMING IMAGE 


YES 



28 


EP 0 516 371 A2 


FIG. 25 


(start ) 


Sll 


N= M - I 


S/2 


k = O 


k=k «■ / 

r 


sSI4 


PREPARE I MAGE OF 
USER k 


515 


DISPLAY IMAGE A T 
POSITION DECIDED BY 
DECIDING PART 66 


YES 



29 


EP 0 516 3*71 A2 


# 


FIG. 26 



S2I 


NO 


I = 

0 




■ 

I* / 



S24 


S25 


DETERMINE DISPLAY POSITION 
OF ITH USER TO RIGHT 


YES 



S2G 


N2 = (M-2)/2 


S27 


S28 



1= /+ / 


I 


S3I 


DETERMINE DISPLAY POSITION 
OF ITH USER TO LEFT 


YES 



S32 


Q END ^) 


30 


